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BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to image processing, and more particularly to an 
advantageous object extraction device, object extraction method, and recording media 
storing an object extraction program used in the extraction of an object image. 

Description of the Related Art 

The object extraction methods described below are known conventional 
methods for obtaining an object extraction image by extraction only an object from a 
plurality of photographed images in which the same object is photographed along with a 
background. 

First, when the photograph image (referred to hereinafter as the "central 
image") of the object that will serve as the object extraction image and the photograph 
images (referred to hereinafter as the "reference images") of the object having parallax 
with respect to the central image are output, the central image is partitioned into bands in 
the horizontal direction, and the image partitioned into bands is fiirther partitioned into 
segments that are enclosed by the edge points of the object. Next, the location of the 
corresponding points that correspond to the end point of each of the segments are 
searched for on an epipolar line, and as a result the depth is estimated based on the 
obtained parallax. Next, based on this estimated depth information, each of the 
segments of the central image is partitioned into foregroimd segments and background 
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segments. Next, the set of partitioned segments in the foreground is obtained as the 
object extraction image. However, in the case that one part of this image is lost when 
this object retraction image is formed, if there is an eflFective segment in a range 
sufficiently close when viewed vertically, the lost segment is interpolated by pixels in 
5 the horizontal and vertical directions. 

Moreover, in this object extraction method, the calculation parameters such as 
the number of pixel spans during each of the calculations and the brightness difference 
used for edge identification is set in advance. 

Furthermore, in the case that a background part is extracted as the object, the set 
oj 10 of segments grouped in the background segments may serve as the object extraction 

Rl image. 

El Fig. 1 5 is a block diagram showing an example of the structure of a 

conventional object extraction device. In this figure, reference numeral 1 is the central 
f^] camera that photographs the central image PI, reference numerals 2-1 and 2-2 are 

^ 15 reference cameras that photograph reference images P2-1 and 2-2, and reference 

numeral 3 is an object extraction calculating unit that finds the object extraction image 
P3 by carrying out an extraction calculation of the object image on the input central 
image PI, and reference images P 2-1 and 2-2 according to the above-described object 
extraction method by using a calculation parameter set in advance. In addition, this 
20 object extraction calculating unit 3 is a conventional object extraction device. 

Moreover, the conventional object extraction device is realized by large-scale 
computers, etc., such as a work station into which an object extraction program that uses 
the above-described convention extraction method, for example, is input. 

However, in the above-described object extraction device, in the case that a part 
25 of the object extraction image is lost, in the interpolation processing using pixels in the 
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horizontal and vertical directions, there is the problem that the locations that should be 
background image are mistaken for a foreground segment and made a part of the object 
extraction image, or an object extraction image expanded beyond the actual object is 
formed, a location that should be a foreground object extraction image is mistakenly 
5 made a part of the background image, and an object extraction image having a blank part 
is formed. 

In addition, when the object extraction calculation is carried out by changing to 
a more sophisticated calculation parameter in order to resolve the above-described 
problems, the amount of the calculating increases, and more time is consumed to find the 

fti 10 object extraction image. As a result a large-sale computer is necessary for realizing the 

nl conventional object extraction device. 

SUMMARY OF THE INVENTION 
^5 In consideration of the above-described problems, an object of the present 

15 invention is to provide an image extraction device, an object extraction method, and a 
recording media that stores an object extraction program wherein an object extraction 
image that better matches the object can be obtained and the amount of computing for 
finding the object extraction image can be decreased. 

In order to resolve the above-described problems, a first aspect of the present 
20 invention provides a first object extraction calculating device that finds the object 

extraction image by carrying out object extraction calculations for extraction an object 
by using a predetermined first calculation parameter on a plurality of photographed 
images having a parallax with respect to the same object, and an incorrect outline 
extraction processing device that extracts an outline fi"om an object extraction image 
25 found by the first object extraction calculating device and extracts as an incorrect outline 
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a straight line segment having a length exceeding a predetermined threshold value within 
the extracted outline. 

In a second aspect of the invention disclosed in the first aspect, the incorrect 
outline extraction processing device provides an outline extraction device that extracts 
an outline from an object extraction image found by the object extraction calculating 
device, an edge pixel calculating device that finds the edge part of the object fi-om a 
predetermined photographed image fi*om among the plurality of photographed images, 
and an incorrect outline extraction device that extracts as an incorrect outline the straight 
line segment within the outline extracted by the outline extraction device that is the 
outline part that does not include the edge part found by the edge pixel calculating 
device and has a length exceeding a predetermined value. 

In a third aspect of the invention disclosed in the first aspect, the incorrect 
outline processing device provides an outline extraction device that extracts an outline 
fi-om an object extraction image found by the object extraction calculating device, an 
edge pixel calculating device that finds the edge part of the object from a specified 
photographed image from among a plurality of photographed images, and an incorrect 
outline extraction device that extracts as an incorrect outline the straight line segment 
that is the outline part within the outline extracted by the outline extraction device that 
does not include the edge part found by the edge pixel calculating device and has a 
length exceeding a predetermined value. 

In a fourth aspect of the invention disclosed in the first aspect, the incorrect 
outline processing device provides an outline extraction device that extracts an outline 
from an object extraction image found by the object extraction calculating device, an 
edge pixel calculating device that finds the edge part of the object from a specified 
photographed image from among a plurality of photographed images, and an incorrect 
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outline extraction device that extracts as an incorrect outline the straight line segment 
that is the outline part within the outline extracted by the outline extraction device that 
does not include the edge part found by the edge pixel calculating device and has a 
length exceeding a predetermined first threshold value and the straight line segment that 
5 is the outline part that intersects the edge part found by the edge pixel calculating device 
and has a length exceeding a predetermined second threshold value. 

In a fifth aspect of the invention disclosed in the first aspect, the straight line 
segment is a straight line segment along the scanning direction of a predetermined 
f=i photographed image. 

SI 10 A sixth aspect of the invention discloses in the first aspect provides a 

h= recalculated region determining device that detennines as the recalculated region the 

partial region that includes the incorrect outline extracted by the incorrect outline 
extraction processing device within the image region of a predetermined photographed 
L[ image, and a second object extraction calculating device that finds the re-extracted 

gi 15 image that is the object extraction image of the recalculated region by carrying out 

object extraction calculation in order to eliminate the incorrect outline of the recalculated 
region by using a second calculation parameter that is different from the first calculation 
parameter on a plurality of photographed images. 

In a seventh aspect of the invention, the recalculated region in the sixth aspect 
20 of the invention is determined as the rectangular region having a predetermined range 
that includes the incorrect outline. 

In an eighth aspect of the invention, the recalculated region determining-device 
in the sixth aspect of the invention re-determines as the recalculated region the region 
that encompasses all of the overlapping recalculated regions instead of a plurality of 
25 overlapping recalculated regions. 



In a ninth aspect of the invention, the recalculated region in the eighth aspect of 
the invention that has been re-determined is determined as the rectangular region having 
the smallest area, 

A tenth aspect of the invention disclosed in the sixth aspect provides an image 
reconstructing device that reconstructs the object extraction image based on the object 
extraction image found by the first object extraction calculating device and the 
re-extracted image found by the second object extraction calculating device. 

In an eleventh aspect of the invention, the image reconstructing device in the 
tenth aspect reconstructs the object extraction image by exchanging the image in the 
region corresponding to the recalculated region within the image region of the object 
extraction image found by the first object extraction calculating device and the 
re-extracted image found by the second object extraction calculating device. 

In a twelfth aspect of the invention, the second calculation parameter in the 
eleventh aspect is a calculation parameter that is used to carry out an object extraction 
calculation that is more sophisticated than the first calculation parameter. 

In a thirteenth aspect of the invention, the object extraction device in the 
eleventh aspect is realized by a small-scale computer. 

In the fourteenth aspect of the invention, the plurality of photographed images 
in the first aspect are photographed by a plurality of cameras that photograph the same 
object fi-om different directions. 

A fifteenth aspect of the invention provides an object extraction calculating 
device that repeats the object extraction calculation for eliminating an incorrect outline 
fi*om a predetermined partial region on the plurality of photographed images having 
parallax with respect to the same object using a predetermined calculation parameter that 
is different fi-om the initial calculation parameter, and finds the re-extracted image. 
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which is the object extraction image of this partial region. 

In a sixteenth aspect of the invention, the partial region in the fifteenth aspect of 
the invention is a region that includes the outline part determined to be an incorrect 
outline in the image region of the object extraction image found by carrying out the 
5 object extraction calculation for extraction the object using the initial calculation 
parameter. 

A seventeenth aspect of the invention in the fifteenth aspect provides an image 
reconstructing device that reconstructs the object extraction image by exchanging the 
image of the partial region in the object extraction image found by the object extraction 

10 calculation using the initial calculation parameter and the re-extracted image. 

An eighteenth aspect of the invention provides a first process that finds an 
object extraction image by carrying out an object extraction calculation for extraction the 
object using a predetermined first calculation parameter on the plurality of photographed 
images that have parallax with respect to the same object, a second process that extracts 

15 the outline from the object extraction image found by the first process, a third process 
that extracts as an incorrect outline the straight line segment within the outline extracted 
by the second process that has a length exceeding a predetermined threshold value, a 
fourth process that determines as the recalculated region the partial region that includes 
the incorrect outline extracted by the third process within the image region of the 

20 predetermined photographed image, a fifth process that finds the re-extracted image, 
which is the object extraction image in the recalculated region, by carrying out object 
extraction calculation for eliminating the incorrect outline in the recalculated region 
using a predetermined second calculation parameter that is different from the first 
calculation parameter on the plurality of photographed images, and a sixth process that 

25 reconstructs the object extraction image based on the object extraction image found by 
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the first process and the re-extracted image found by the fifth process. 

In a nineteenth aspect of the invention, the third process in the eighteenth aspect 
finds the edge part of the object fi-om the predetermined photographed image from 
among a plurality of photographed images and extracts as an incorrect outline either or 
5 both a straight line segment within the outline extracted by the second process that is the 
outline part not included in the found edge part and has a length exceeding a 
predetermined first threshold value or a straight line segment that is the outline part that 
intersects the found edge part and has a length exceeding a predetermined second 
^: threshold value. 

10 In a twentieth aspect of the invention, a computer executes a first process that 

HI finds an object extraction image by carrying out object extraction calculation for 

B extraction the object using a predetermined first calculation parameter on a plurality of 

photographed images having parallax with respect to the same object, a second process 

I y 

ry. that extracts an outline from the object extraction image found by the first process, and a 

15 third process that extracts as an incorrect outline the straight line segment within the 
outline extracted by the second process that has a length exceeding a predetermined 
threshold value. 

In a twenty-first aspect of the invention, the object extraction program in the 
twentieth aspect of the invention executes on a computer a fourth process that 

20 determines as the recalculated region the partial region that includes the incorrect outline 
extracted by the third process within an image region of the predetermined photographed 
image, a fifth process that finds the re-extracted image, which is the object extraction 
image in the recalculated region, by carrying out object extraction calculation for 
eliminating the incorrect outline in the recalculated region using a predetermined second 

25 calculation parameter that is different fi-om the first calculation parameter on the 
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plurality of photographed images, and a sixth process that reconstructs the object 
extraction image based on the object extraction image found by the first process and the 
re-extracted image found by the fifth process. 

In a twenty-second aspect of the invention, the object extraction program in the 
5 twentieth aspect of the invention fiirther executes on a computer a process that, in the 
third process, finds the edge part of the object fi-om a predetermined photographed image 
ft-om among a plurality of photographed images, and a process that extracts as incorrect 
outlines either or both of a straight line segment that is the outline part within the outline 

CI extracted by the second process not included in the found edge part and that has a length 

4) 

2:1 10 exceeding a predetermined first threshold value and a straight line segment that is the 

^] outline part that intersects the foimd edge part and has a length exceeding a 

5 J predetermined second threshold value. 

Ql A twenty-third aspect of the invention executes on a computer a process that 

hi: finds the re-extracted image, which is the object extracted image of the partial region, by 

Cl 15 repeating the object extraction calculation for eliminating incorrect outlines within a 

predetermined partial region on the plurality of photographed images having parallax 
with respect to the same object using a predetermined calculation parameter that is 
different fi-om the initial calculation parameter. 

In a twenty-fourth aspect of the invention, the object extraction program in the 
20 twenty-third aspect of the invention further executes on a computer a process that 

determines in a partial region the region that includes the determined outline part as an 
incorrect outline in an image region of the object extraction image found by carrying out 
the object extraction calculation for extraction the object using a initial calculation 
parameter 

25 In a twenty-fifth aspect of the invention, the object extraction program in the 



10 



twenty-third aspect of the invention further executes on a computer a process that 
reconstructs the object extraction image by exchanging the image of the partial region in 
the object extraction image found by the object extraction calculation using the initial 
calculation parameter and the re-extracted image. 

5 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block drawing showing the structure of the object extraction 
apparatus 101 according to the first embodiment of the present invention. 

Fig. 2 is a block drawing showing the structure of the object extraction device 
10 102 according to the first embodiment of the present invention. 

Fig. 3 is a block drawing showing the structure of the object extraction device 
1 03 according to the first embodiment of the present invention. 

Fig. 4 is a first drawing for explaining the operation of the object extraction 
device 101 shown in Fig. 1. 
15 Fig. 5 is a second drawing for explaining the operation of the object extraction 

device 101 shown in Fig. 1. 

Fig. 6 is a third drawing for explaining the operation of the object extraction 
device 101 shown in Fig. 1. 

Fig. 7 is a fourth drawing for explaining the operation of the object extraction 
20 device 101 shown in Fig. 1. 

Fig. 8 is a fifth drawing for explaining the operation of the object extraction 
device 101 shown in Fig. 1. 

Fig. 9 is a first drawing for explaining the operation of the object extraction 
device 102 shown in Fig. 2. 
25 Fig. 10 is a second drawing for explaining the operation of the object extraction 
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device 102 shown in Fig. 2. 

Fig. 1 1 is a third drawing for explaining the operation of the object extraction 
device 102 shown in Fig. 2. 

Fig. 12 is a fourth drawing for explaining the operation of the object extraction 
5 device 102 shown in Fig. 2. 

Fig. 13 is a fifth drawing for explaining the operation of the object extraction 
device 102 shown in Fig. 2. 

Fig. 14 is a sixth drawing for explaining the operation of the object extraction 
device 102 shown in Fig. 2. 
10 Fig. 1 5 is a block drawing showing the structure of a conventional object 

extraction device. 

DETAILED DESCRIPTION OF THE INVENTION 
The embodiments of the present invention will be explained referring to the 

15 figures. 

Fig. 1 is a block diagram showing the structure of the object extraction device 
101 according to a first embodiment of the invention. In the figures, reference numeral 
1 is a center camera that photographs a photographed image PI (hereinafter, referred to 
as a "center image") of an object that v^U serve as the object extraction image, reference 

20 numerals 2-1 and 2-2 are reference cameras that photographs ft*om different directions 
the same object that is photographed by the center camera, and photographs 
photographed images P2-1 and P2 (hereinafter, referred to as the reference images) of 
the object that have parallax with respect to the center image PI, and reference numeral 
3 is a first object extraction calculating unit that carries out an extraction calculation of 

25 the object image using a predetermined calculation parameter on the center image PI 



12 

and the reference images P2-1 and 2-2 that have been mput, and finds the object 
extraction image P3. Moreover, the center camera 1, the reference cameras 2-1 and 2-2, 
and the first object extraction calculating unit 3 are structures similar to the conventional 
object extraction device shown in Fig. 15, the object extraction calculation of the first 
object extraction calculating unit 3 is like the conventional object extraction calculating 
unit 3 in Fig. 15, and the object extraction calculation of the first object extraction 
calculating unit 3 is carried out using the first calculation parameter. 

Reference numeral 111 is an incorrect outline extraction processing unit that 
extracts firom the object extraction image P3 found by the first object extraction 
calculating unit 3 a suspected incorrect outline as the outline of the object 3, and 
comprises an outline extraction unit 4 that extracts an outline by tracing the outline part 
of an object from the object extraction image P3, an edge pixel calculating unit 5 that 
extracts the pixels on the edge part of the object from the input center image PI , and a 
first incorrect outline extraction unit 6 that extracts as an incorrect outline the straight 
line segment in the outline extracted by the outline extraction unit 4 that is the straight 
line segment not included in the pixels of the edge part found by the edge pixel 
calculating unit 5, and has a length exceeding a predetermined threshold value. 

Reference numeral 7 is a recalculated region determining unit that determines 
as a recalculated region the rectangular region in a predetermined range that includes the 
incorrect outline extracted by the first incorrect outline extraction unit 6, reference 
numeral 8 is a second object extraction calculating unit that finds the re-extracted image 
P4, which is the object extraction image of the recalculated region, by carrying out an 
object extraction calculation for the recalculated region determined by the recalculated 
region determining imit 7 using a predetermined second calculation parameter that is 
different firom the first calculation parameter on the input center image PI and the 
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reference images P2-1 and 2-2, and reference numeral 9 is an image reconstructing unit 
that reconstructs the object extraction image P5 by exchanging the image of the region 
that corresponds to the recalculated region within the image region of the object 
extraction image P3 found by the first object extraction calculating unit 3 and the 
5 re-extracted image 4 found by the second object extraction calculating device 8. 

Moreover, in the object extraction calculation of the second object extraction calculating 
unit 8, the object extraction calculation is carried out by using a second calculation 
parameter, like the first object extraction calculating unit 3. 

Next, Fig. 4 to Fig. 8 are an example of drawings for explaining the operation 

ri 10 of the object extraction device 101 shown in Fig. 1. The operation of the object 

ill 

extraction device 101 will be explained referring to these drawings. 
m First, Fig. 4 shows the outline Wl of an object photographed by the center 

= camera 1, and the outline Wl of this complete circle and the interior of this outline Wl 

are the object. First, when the center image PI and the reference images P2-1 and 2-2, 

: - ; 

Jir 15 which are the object photographed by the center camera 1 and the reference cameras 2-1 

and 2-2, are input into the object extraction device 101, the first object extraction 
calculating unit 3 carries out object extraction calculation using a first calculation 
parameter, and the object extraction image P3 shown in Fig. 5 is found, where in Fig. 5, 
the solid lines between points Al - A2, A2 - A3, A3 - A4, and A4 - Al and the interior 

20 enclosed by these lines is the object extraction image P3. 

Next, from the object extraction image P3 in Fig. 5, the outline extraction part 4 
extracts as the outlines of the object the outline W2 between points Al - A4, which is 
part of the complete circle; outline W3, which is the straight line segment between 
points Al - A2; outline W4, which is the straight line segment between points A2 - A3; 

25 and outline W5, which is the straight line segment between points A3 - A4. However, 
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outline W2 is a part of the outline Wl, and outlines W3 - W5 are not included in outline 
Wl . In addition, the edge pixel calculating unit 5 extracts from the input center image 
PI the pixels of the outline Wl shown in Fig. 4 as the edge part of the object. 

Next, among the outlines W2 - W5 extracted by the outline extraction unit 4 
5 and among outlines W3 - W5 that do not include the pixels of outline Wl in the edge 
part found by the edge pixel calculating unit 5, the first incorrect outline extraction unit 6 
extracts as incorrect outlines the outlines W3 and W5, which are straight line segments 
that have a length exceeding a predetermined threshold value, where this predetermined 
threshold value is, for example, 8 pixels. Moreover, in the case that an incorrect outline 

10 is not extracted, the processing ends, and the object extraction image P3 is made the • 
object extraction image P5 by the image reconstructing xmit 9. 

Next, the re-calculated region determining unit 7 determines as one recalculated 
region the rectangular region SI having the predetermined range shown in Fig. 6, which 
includes the outline W3 that was extracted as an incorrect outline by the incorrect outline 

15 extraction unit 6, and determines as one recalculated region the rectangular region S2 
having the predetermined range shown in Fig 6, which includes the outline W5. Here, 
for the straight line segment in a horizontal direction, the predetermined range of this 
determined rectangular region is, for example, the range separated only by the distance 
of 20 pixels each above, below, left, and right that enclose the line. In contrast, for a 

20 straight line segment in the vertical direction, the range is separated only by the distance 
of 20 pixels each above and below and 40 pixels each left and right that enclose this line. 
Next, the recalculated region determining unit 7 determines as the recalculated region 
the rectangular region S3 that is the rectangular region that encompasses all of the 
overlapping recalculated region shown in Fig. 7, and that has the minimum area. 

25 Next, the second object extraction calculating unit 8 carries out the object 
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extraction calculation for the rectangular region S3, which is the recalculated region 
determined by the recalculated region determining imit 7 using a predetermined second 
calculation parameter, on the input center image P and the reference images P2-1 and 
2-2, and finds as the re-extracted image P4 the outline W6 shown in Fig. 8 and the 
5 inclined part encompasses by this outline. 

Next, the image reconstructing imit 9 exchanges the image in the region 
corresponding to the recalculated region in the image region of the object extraction 
image P3 in Fig. 5, that is, the image of the rectangular region S3 in Fig. 7, with the 
image of the rectangular region S3 in Fig. 8. By this image exchange, the image of the 

10 inclined part in Fig. 7 and the inclined part in Fig. 8 are combined, and the object 
extraction image P5 having the outline Wl in Fig. 4 is reconstructed. 

Next, Fig. 2 is a block diagram showing the structure of the object extraction 
device 102 according to the second embodiment of the present invention. In this figure, 
only the second incorrect outline extraction unit 1 1 in the incorrect outline extraction 

15 processor 112 differs from the object extraction device 101 in Fig. 1. This second 

incorrect outline extraction unit 1 1 extracts as an incorrect outline the straight part that is 
the outline part within the outline extracted by the outline extraction unit 4that intersects 
the pixels of the edge part found by the edge pixel calculating unit 5, and has a length 
exceeding a predetermined threshold value. 

20 Fig. 9 to Fig. 14 are one example of drawings for explaining the operation of 

the above-described object extraction device 102, and referring to these figures, the 
operation of the object extraction unit 102 will be explained below. 

First, Fig. 9 shows the outline Wl 1 of an object photographed by the center 
camera 1, and the outline Wl 1 of the complete circle and interior of the outline Wl 1 

25 comprise the object. In the beginning, the center image 1 and the reference images 
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P2-1 and 2-2, which are the object photographed by the center camera 1 and the 
reference cameras 2-1 and 2-2, are input into the object extraction device 102, and the 
object extraction image P3 shown in Fig. 10 is found by the first object extraction 
calculating unit 3 using the first calculation parameter, where in Fig. 1 0, the solid lines 
5 between points Bl - B2, B2 - 83, and B3 - Bl and the interior surrounded by the solid 
lines are the object extraction image P3. 

Next, from the object extraction image in Fig. 10, the outline extraction unit 4 
extracts as the outline of the object the outline W12 between points Bl - B3, which are 
^\ one part of the complete circle, the outline W13, which is the straight line segment 

S| 10 between points Bl - B2, and the outline W14, which is the straight line segment 
H between points B2 and B3. However, outline W12 is a part of the outline Wl 1 , and 

outlines Wl 3 and W14 are not included in outline Wl 1 . In addition, the edge pixel 
~. calculating unit 5 extracts the pixels in the edge waveforms W15 and W16 shown in Fig. 

fi i 

ij: 1 1 as the edge part of the object from the input center image P 1 , where the edge 

ri 15 waveform 1 5 is equal to the outline Wl 1 . 

Next, the second incorrect extraction imit 1 1 extracts as an incorrect outline 
both the outlines W13 and W14 that are straight line segments that have lengths that 
exceed a predetermined threshold value in the outlines W13 and W14 that intersect the 
pixels of the edge waveform W16 (shown by the broken line in Fig. 10) found by the 
20 edge pixel extraction unit 5 among the outlines W12 to W14 that were extracted by the 

outline extraction unit 4. Moreover, in the case that an incorrect outline is not extracted, 
the processing stops, and the object extraction image P3 is made the object extraction 
image P5 by the image reconstructing unit 9. 

Next, like the above-described first embodiment, the recalculated region 
25 determining unit 7 determines as the recalculated region the rectangular region S 13, 
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shown in Fig. 13, that has the smallest area, and is the rectangular region that 
encompasses the rectangular region SI 1 having the predetermined range shown in Fig. 
12 that includes the outline W13 that was extracted as an incorrect outline by the first 
incorrect outline extraction unit 6 and the rectangular region S12 that has a 
predetermined range including outline W14. 

Next, the second object extraction calculating unit 8 carries out the object 
extraction calculation for the rectangular region S13 using a predetermined second 
calculation parameter on the input center image PI and the reference images P2-1 and 
2-2, and finds as the re-extracted image P4 the outline W17, shown in Fig. 14, and the 
inclined part enclosed by this outline. 

Next, like the above-described first embodiment, the image reconstructing part 
9 combines the image of the inclined part in Fig. 13 and the inclined part in Fig. 14 by 
exchanging the image of the rectangular region SI 3 in Fig. 13 and the image of the 
rectangular region SI 3 in Fig. 14, and thereby reconstructs the object extraction image 
P5 having the outline Wll shown in Fig. 9. 

Next, Fig. 3 is a block diagram shoving the structure of the object extraction 
device 103 according to the third embodiment of the present invention. In this figure, 
the structure differs from the above-described embodiments in that the incorrect outline 
exacting processor 113 has a first incorrect extraction unit 6 of the first embodiment and 
the second incorrect outline extraction unit 1 1 of the second embodiment, and the 
recalculated region determining unit 1 2 determines as the recalculated region the 
rectangular region having a predetermined range that includes this incorrect outline for 
the incorrect outline extracted by the first and second incorrect outline extraction units 6 
and 1 1 . Therefore, the region that becomes the recalculated region extracted as the 
incorrect outline by the object extraction device 103 is either one or a plurality of the 
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rectangular regions in the rectangular region having the smallest area that is the 
rectangular region having a predetermined range, which is the outline part that does not 
include the pixels in the edge part found by the edge pixel calculating unit 5, and 
includes a straight line segment that has a length exceeding a predetermined first 
5 threshold value, the rectangular region that has a predetermined region that is the outline 
part that intersects the pixels of the edge part and includes a straight line segment that 
has a length exceeding a predetermined second threshold value, and a rectangular region 
that encompasses all of the overlapping recalculated region. Moreover, the structure of 
the other blocks and their operation is identical to that of the above-described 
10 embodiments. 

Moreover, in the above-described embodiments, if the first and second incorrect 
outline extraction units 6 and 1 1 extract as the incorrect outline the straight line segment 
along the scanning direction of the center image PI, then the amount of calculation 
expended in the incorrect outline extraction can be reduced. 

15 Moreover, in the above-described embodiments, because the recalculated region 

determining units 7 and 12 re-determine as the recalculated region the rectangular region 
that includes all of the overlapping recalculated region and has the smallest area, the 
redundant parts being calculated several times can be avoided, and the amount of 
calculation of the second object extraction calculating unit 8 can be reduced. 

20 Moreover, in the above-described embodiments, the recalculated region 

determining units 7 and 12 determine as the rectangular region the recalculated region, 
but the shape of the recalculated region is not limited to being a rectangle. 

Moreover, a program for reading each of the parts shown in Figs. 1,2, and 3 
can be stored on computer readable recording media, and the object extraction 

25 processing carried out by reading the program stored on these recording media into a 
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computer system and executing the program. Moreover, here, "computer system" 
includes any OS and hardware such as peripheral devices, hi addition, in the case that 
the WWW system is used, "computer system" includes any home page providing 
environment (or display environment). In addition, "computer readable recording 
5 media" denotes memory media such as transferable media, examples being floppy disks, 
opto-magnetic disks, ROMs, and CD-ROMs, and hard disks built-into computer 
systems. 

Furthermore, "computer readable recording media" includes items (such as 
transmitting media and transmission waves) that dynamically maintain the program for a 

J] 10 short time interval, such as a communication line used when transmitting the program 

01 

via telecommunication lines such as networks and telephone lines, an example being the 
Oi Intemet, and in this case, items which maintain the program temporarily, such as the 

^ volatile memory in a computer system serving as a server or client. In addition, the 

f^;- above-described program can realize one part of the functions described above, and 

2; 15 furthermore, the above-described functions can be realized by a combination of 

programs, that is, sub-files (or sub-programs), that are already stored in a computer 

system. 

Moreover, in the above-described embodiments, the second object extraction 
calculating unit 8 can compare the second calculation parameter used in the object 

20 extraction calculation v^th the first calculation parameter used by the first object 

extraction calculating unit 3, and by making the calculation parameter carry out a more 
sophisticated- object extraction calculation, can correct the incorrect outline generated by 
the first object extraction calculating unit 3 by carrying out a more sophisticated 
calculation on the part suspected of being an incorrect outline. 

25 Furthermore, compared to the case of using a calculation parameter for carrying 



20 



out a more sophisticated object extraction calculation in the conventional object 
extraction device, carrying out a more sophisticated calculation is not limited to the part 
suspected of being an incorrect outline, and thus the overall amount of calculation of the 
object extraction can be decreased. As a result, the object extraction device according 
5 to the above-described embodiments of the present invention can be carried out by a 
small-scale computer such as a personal computer rather than a large-scale computer, as 
is done conventionally. 

As explained above, the present invention finds the object extraction image by 
carrying out object extraction calculation using a predetermined first calculation 

10 parameter on a plurality of photographed images having parallax with respect to the 
same object, extracts an outline fi*bm this object extraction image, and extracts as an 
incorrect outline a straight line segment that has a length exceeding a predetermined 
threshold value within this extracted outline. In addition, by determining as a 
recalculated region the partial region that includes this extracted incorrect outline and 

15 carrying out object extraction calculation of the recalculated region using a 

predetermined second calculation parameter that differs form the first calculation 
parameter, finding the re-extracted image, which is the object extraction image of the 
recalculated region, and reconstructing the object extraction image based on the object 
extraction image found using this first calculation parameter and the re-extracted image, 

20 . in comparison to the conventional art, an object extraction image can be obtained that 
matches the object. 



